Comments for MEDB 5501, Week 8

Review multiple comparisons issue

• Type I error: rejecting the null hypothesis when the null hypothesis is true.
  • Multiple simultaneous hypotheses increase the Type I error rate.

Bonferroni inequality for two simultaneous hypotheses

Define

• \(E_1\) = Type I error for Hypothesis 1
• \(E_2\) = Type I error for Hypothesis 2
  • \(P[E_1\ \cup\ E_2]=P[E_1]+P[E_2]-P[E_1\ \cap\ E_2]\)
  • \(P[E_1\ \cup\ E_2]\ \le\ P[E_1]+P[E_2]\)
  • \(P[E_1\ \cup\ E_2]\ \le\ P[E_1]+P[E_2]\)
  • \(P[E_1\ \cup\ E_2]\ \le\ 2 \alpha\)

Bonferroni adjustment

• For m hypotheses
  • \(P[E_1\ \cup\ ...\ \cup E_m]\ \le\ m \alpha\)
• Test each hypothesis at \(\alpha/m\)
  • Preserves overall Type I error rate
• Example, 3 simultaneous hypotheses
  • Reject H0 if p-value < 0.0133

Controversies over Bonferroni adjustment

• Increases Type II errors
• Impractical for large values of m
• Works poorly for highly correlated tests
• Ambiguity in definition of “simultaneous” hypotheses

Alternatives to Bonferroni adjustment

• False discovery rate
• Designation of primary and secondary outcomes
• Subjective assessment of simultaneous hypotheses

Review two-sample t-test

• \(H_0:\ \mu_1=\mu_2\)
• \(H_1:\ \mu_1 \ne \mu_2\)
  • Or a one tailed alternative
• \(T=\frac{\bar X_1-\bar X_2}{SE(\bar X_1-\bar X_2)}\)
  • Accept H0 if T is close to zero.

What to do with three or more groups?

• \(H_0:\ \mu_1=\mu_2=...=\mu_k\)
• \(H_1:\ \mu_i \ne \mu_j\) for some i, j
  • Note: one-tailed test is tricky.
• Accept H0 if the F ratio (defined below) is close to 1.

Important assumptions

• Same as independent-samples t-test
  • Normality
  • Equal variances
  • Independence

How to check assumptions

• Boxplots
• Analysis of residuals, \(e_{ij}\)
  • \(e_{ij}\)= Observed - Predicted
  • \(e_{ij}= Y_{ij}-\bar{Y}_i\)

Tukey post hoc tests

• If you reject H0, which values are unequal
  • With k groups, there are k(k-1)/2 comparisons
• Studentized range (Tukey test)
  • Requires equal sample sizes per group
  • Uses harmonic mean approximation for unequal sample sizes.
    • Do not use harmonic means if seriously different sample sizes.

Alternatives to Tukey post hoc tests

• Bonferroni adjustment
  • Works for unequal sample sizes per group
  • Works for unequal variances
• Dunnett’s test
  • Treatment versus multiple controls
• Scheffe’s test
  • Works for complex comparison
    • Example \(\mu_1\ vs.\ \frac{\mu_2+\mu_3+\mu_4}{3}\)

Artificial data

   g  y
1  1 23
2  1 30
3  1 25
4  2 33
5  2 36
6  2 41
7  2 37
8  2 43
9  3 24
10 3 29
11 3 31

Scatterplot

Total SS = 452

Within SS = 116

Between SS = 336

Degrees of freedom

• For Total SS, df = 10
• For Within SS, df = 8
• For Between SS, df = 2
• In general,
  • N = number of observations
  • k = number of groups
    • Total df = N-1
    • Within df = N-k
    • Between df = k-1

ANOVA table, 1 of 4

ANOVA table, 2 of 4

ANOVA table, 3 of 4

ANOVA table, 4 of 4

ANOVA table from the general linear model

Regression approach to ANOVA, 1 of 4

Regression approach to ANOVA, 2 of 4

Regression approach to ANOVA, 3 of 4

Regression approach to ANOVA, 4 of 4

Predicted values and residuals, 1 of 2

Predicted values and residuals, 2 of 2

Recode into different variables dialog box

Old and new Values dialog box

General Linear Model | Univariate dialog box

Descriptives output

Boxplot

Analysis of variance table

Analysis of variance table with irrelevant rows removed

Parameter estimates

Tukey post hoc test, 1 of 7

Tukey post hoc test, 2 of 7

Tukey post hoc test, 3 of 7

Tukey post hoc test, 4 of 7

Tukey post hoc test, 5 of 7

Tukey post hoc test, 6 of 7

Tukey post hoc test, 7 of 7

Checking assumptions, boxplots

Checking assumptions, descriptive statistics

Residual analysis, 1 of 2

Residual analysis, 2 of 2

Analysis of school year, 1 of 4

Analysis of school year, 2 of 4

Analysis of school year, 3 of 4

Analysis of school year, 4 of 4